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REMARKS 

1. Objection to the specification: 

5 Alt informalities identified by the Examiner and additional informalities identified in a 
thorough review of the specification have been corrected. This includes eliminating extra 
white-space and increasing the size of the equations. The changes are self-evident. No 
new matter is entered. A likely source of these errors has been identified as the electronic 
filing system. 

10 

A copy showing changes made and a clean copy are provided. 
Reconsideration of the specification is respectfully requested. 
15 2. Claim objections: 

In claim 2, the term "integer-natural" is revised to "natural". Claim 1 1 has been cancelled. 
Zero and negative scaling factors are not consistent with the disclosure. 

20 In claims 1 and 10, "comprising following steps" is revised to "comprising the following 
steps." 

In claim 10, "portion data" is revised to "portion of data." 

25 No new matter is entered by any of these amendments. Withdrawal of these objections is 
respectfully requested. 

3. Rejection of claims 2 and 11 under 35 U.S.C. 112, second paragraph: 

22 
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In claim 2, Natural number 5 ' is now recited. Claim 1 1 has been cancelled. 



In claim 2, steps (a) and (b) are identified so as to make sense. 



5 No new matter is entered by any of these amendments. Withdrawal of this rejection is 
respectfully requested. 

4. Rejection of claims 1 and 8 under 35 U.S.C. 103(a) as being unpatentable over 
Nacman (US PGPub 2003/0128896) in view of Nishihaki et ah (JP 2000 11 48997 A): 



10 



In the "Summary of Invention" one of the applicant's stated objectives is u to provide a 
method for scaling a digital picture with reduced memory." Accordingly, claim 1 is 
amended to expressly recite "for each different portion in sequence, repeating steps (a) 
and (b) to form the scaled picture" which is meant to further distinguish the serial-nature 
15 of the invention from the parallel-nature of the cited art. This amendment adds no new 
matter. 



The Examiner has asserted that processing images in parallel is functionally equivalent to 
the previously recited "repeating steps." While the end results may be similar, the 
20 methods for obtaining those results are quite different. The amendment to claim 1 is 
meant to emphasize this difference. 

Since an obviousness rejection requires a combination that teaches all limitations of the 
claims, and since neither Nacman nor Nishihaki teach or suggest serially scaling portions 
25 of an image (Nishihaki expressly teaches parallel scaling), this rejection should be 
withdrawn. 

Moreover, given the cited art, one of ordinary skill in the art would not arrive at the 

invention as recited in amended claim 1. The motivation provided by the Examiner 

23 
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("faster image scaling") necessarily results in parallel scaling. One or ordinary skill in the 
art would likely fully recognize the speed advantage of Nishihaki. in fact, they probably 
wouldn't even consider this art if high-speed scaling was not of interest. Thus, the 
applicant argues that there is no motivation to make the combination of Nacman and 
5 Nfcfrihaki to achieve serial scaling as now unequivocally recited in claim 1. Rather, the 
serial scaling concept goes against Nishihajq 's teachings. 

Reconsideration of claims 1 and 8 is requested in view of the amendment to claim 1. 
Claim 8 is dependent and should be allowed if claim 1 is allowed. 

10 

5. Rejection of claims 2-4 under 35 U.S.C. 103(a) as being unpatentable over 
Nacman in view of Nishihaki as applied to claim 1 above, and further in view of Van 
Asma et al. (US 6,600,514 Bl): 

15 Reconsideration of claims 2-4 is requested in view of the amendment to claim 1. Claims 
2-4 are dependent and should be allowed if claim 1 is allowed. 

6. Rejection of claims 5-7 under 35 U.S.C. 103(a) as being unpatentable over 
Nacman in view of Nishihaki as applied to claim 1 above, and further in view of 

20 Nielsen (US 6,591,011 Bl): 

Claims 6 and 7 are amended to narrow the recited general mirroring to a localized 
mirroring that is actually a boundary condition of the filter. The mirror boundary 
condition was recited in the original disclosure and no new matter is entered. This 
25 boundary condition further refines the invention by allowing for higher quality scaled 
images even when buffer size is substantially limited. 

While Nielsen, teaches mirroring images, Nielsen does not teach or suggest a mirror 
boundary condition in a filter. 

24 
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Reconsideration of claims 5-7 is requested in view of the amendments to claims 1, 5, and 
7. Claims 5-7 are dependent and should be allowed if claim 1 is allowed. 

• * 

5 7. Rejection of claim 9 under 35 U.S.C. 103(a) as being unpatentable over Nacman in 
view of Nishihaki as applied to claim 1 above, and further in view of Iga (US PGPub 
2004/0021790 Al): 

Claim 9 is amended to amend the recited removal of part of the picture to omitting part of 
10 the picture, which the applicant asserts more narrowly defines the decimation of extra 
rows/columns of an image in down-scaling recited in the original disclosure. No new 
matter is entered. 

Iga, on the other hand, teaches noise removal, which cannot reasonably be read as 
15 "omitting part of the digital picture." 

Reconsideration of claim 9 is requested in view of the amendments to claims 1 and 9. 
Claim 9 is dependent and should be allowed if claim 1 is allowed. 

20 8. Rejection of claims 10 and 16 under 35 U.S.C. 103(a) as being unpatentable over 
Masaki (US PGPub 2003/0218774 Al) in view of Nacman : 

Claim 10 is amended to be consistent with the amended claim 1, that is, a new step (h) is 
added to emphasize the serial processing aspect of the invention. 

25 

Although without further amendment claim 10 would stand with the amended claim 1 and 

should be patentable over the cited art as claim 1 is asserted to be, claim 10 is further 

amended to emphasize another aspect of the invention. Specifically, the filter used is 

recited as having L taps and the sizes of the portions of data are limited to correspond to L. 

25 
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This was described in the first paragraph of the 'Detailed Description" of the original 
disclosure, thus, no new matter is entered. 

Accordingly, claim 1 1 is cancelled to avoid redundancy. The variable L is preferred over 
5 N in claim 10 since it directly corresponds back to the disclosure. Furthermore, L is 
inherently a natural number since zero or negative sizes or taps are illogical or redundant. 
Hence, the term "natural number" is omitted from claim 10. 

* 

The combination of Masaki and Nacman requires processing by two filters, one for each 
10 direction. There is no suggestion in either art that one filter can be used. Rather, 
Nacman discloses one processor for each direction (i.e. two processors 22 and 24), and 
Masaki discloses two filters (i.e. FIL3 and MAG 5). Thus, having made the combination 
of Nacrnan and Masaki, one of ordinary skill in the art would not recognize that the 
combination could be reduced to a single. filter. 

15 

In addition, there is no suggestion in Asma (from rejection of claim 11) to limit image 
data to specific sizes corresponding to the filter taps. 

Therefore, the amended claim 10 includes limitations that are not taught or suggested by 
20 the cited art, and are further not obvious due to the omission of a second filter. 

These are not trivial differences, in fact, they allow for distinct advantages of the 
invention, namely, to allow "horizontal-scaling and vertical-scaling processes to use the 
same number of filter taps for achieving the best possible quality, while the buffer 
25 memory requirement is kept to be as small as possible and is proportional to the filter 
length." (second last paragraph of "Detailed Description".) 

Reconsideration of claims 10 and 16 is requested in view of the amendment to claim 10. 

Claim 16 is dependent and should be allowed if claim 10 is allowed. 

26 
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9. Rejection of claims 11-13 under 35 U.S.C. 103(a) as being unpatentable over 
Masaki in view of Nacman as applied to claim 10 above, and further in view of 
As ma : 

5 

Claim 1 1 has been cancelled. 

Reconsideration of claims 12 and 13 is requested in view of the amendment to claim 10. 
Claims 12 and 13 are dependent and should be allowed if claim 10 is allowed. 

10 

10. Rejection of claims 14 and 15 under 35 U.S.C. 103(a) as being unpatentable over 
Masaki in view of Nacman as applied to claim 10 above, and further in view of 
Nielsen : 

15 Claims 14 and 15 are amended to narrow the recited general mirroring to a localized 
mirroring that is actually a boundary condition of the buffer and filter. The mirror 
boundary condition was recited in the original disclosure and no new matter is entered. 
This boundary condition further refines the invention by allowing for higher quality 
scaled images even when buffer size is substantially limited. 

20 

While Nielsen teaches mirroring images, Nielsen does not teach or suggest a mirror 
boundary condition in a buffer or filter. 

Reconsideration of claims 14 and 15 is requested in view of the amendments to claims 10, 
25 15, and 15. Claims 14 and 15 are dependent and should be allowed if claim 10 is allowed. 



27 
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Respectfully submitted, 




Date: 



May 9. 2005 



5 Winston Hsu, Patent Agent No. 4 1 ,526 

P.O. BOX 506, Merrifield, VA 221 16, U.S.A. 
Voice Mail: 302-729-1562 
Facsimile: 806-498-6673 . 
e-mail : winstonhsu@naipo.com 

10 

Note: Please leave a message in my voice mail if you need to talk to me. The time in D.C. 
is 12 hours behind the Taiwan time, i.e. 9 AM in D.C. ~ 9 PM in Taiwan). 
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CLEAN COPY OF THE SPECIFICATION 

The entire specification is presented: 
5 Background of Invention 

1 . Field of the Invention 

The present invention relates to scaling digital pictures, and more particularly, to 
scaling digital pictures with reduced memory usage. 

10 

2. Description of the Prior Art 

In practice, scaling digital pictures is usually performed for still images or video 
frames to fit a specific application. The conventional method to scale a picture is to scale 
its horizontal dimension (width) first and then its vertical dimension (height), or vice 
15 versa. This requires a large buffer memory to store the results of scaling the horizontal 
dimension. 

Please refer to Fig. 1 and Fig. 2. Fig. 1 is a schematic diagram of an up-scaling 
process according to the prior art, and Fig. 2 is a schematic diagram of a down-scaling 
20 process according to the prior art W oM and How are the old width and old height; Wnew 

and H ncw are the new width and new height; SRC is a memory for storing a source picture; 
DST is a memory for storing a scaled picture; and BUFFER is a memory for temporarily 
storing data during the scaling process. The method for performing a scaling process is to 
use the sampling formula at a new sampling point as Eq. L 

25 x(t) = ^x(n)h(t - n) (Eq. 1) 

Where x(t) is the new pixel value at the new sampling point t from n=0, x(n) is the 
original pixel value at index n, and h(t-n) is the value of an interpolation function shifted 

n 
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by t from index n. Furthermore, the coefficients i and j give the number of original pixels 
involved in reconstructing x(t), i.e. the number of original pixels involved is given by 
(i+j+1). As a result, based on Eq. 1, the scaling process can be realized by a filter-bank 
implementation. The number (i+j+1) gives the number of filter taps and h(n) is the tap 
5 weighting at n. The sampling process and the theory behind it are well known by those in 
the art, and accordingly it will not be discussed in detail here. 

Ideally, a large buffer memory provides the freedom for a scaling process to exploit 
filters of any length to achieve the required scaling quality. That is, the size of the buffer 

10 memory required is (W ne w*Hoid) bytes. However, from Fig. 1 and Fig. 2, the problem is 
when up-scaling the input picture the memory required increases dramatically. For 
example, if a picture is to be scaled to be two times larger in each dimension, the buffer 
memory required becomes (W ncw *How) = (2* W 0 | d *Hoid). This high memory requirement is 
not feasible in some applications. In particular, this high memory requirement is not 

15 suitable for hardware implementation by integrated circuits (ICs). 

Summary of Invention 

* 

It is therefore a primary objective of the claimed invention to provide a method for 
20 scaling a digital picture with reduced memory to solve the above-mentioned problem. 

According to the claimed invention, a method for scaling a digital picture 

comprising following steps:(a) inputting a source picture to a source memory; (b) 

providing a first buffer and a second buffer; (c) determining scaling factors; (d) 

25 generating initial data in the first buffer and second buffer; (e) transferring a portion data 

of the digital picture from the source memory to the first buffer; (0 using a fitter to scale 

the data in the first buffer in a first direction and storing the scaled data in the second 

buffer; (g) using the filter to scale the data in the second buffer in a second direction and 

storing the scaled data in a destination memory; and (h) outputting a scaled picture from 

12 
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the destination memory. 



These and other objectives of the present invention will no doubt become obvious to 
those of ordinary skill in the art after reading the following detailed description of the 
5 preferred embodiment that is illustrated in the various figures and drawings. 

Brief Description of Drawings 

Fig. 1 is a schematic diagram of an up-scaling process according to the prior art. 
10 Fig. 2 is a schematic diagram of a down-scaling process according to the prior art. 

Fig. 3 is a schematic diagram of an up-scaling process according to the present invention. 
Fig. 4 is a schematic diagram of a down-scaling process according to the present 

* 

invention. 

Fig. 5 is a flowchart according to the present invention. 
1 5 Fig. 6 is a flowchart of a preferred embodiment according to the present invention. 
Fig. 7 is a schematic diagram of mirrored data extension. 



Detailed Description 

20 Please refer to Fig. 3 and Fig. 4. Fig. 3 is a schematic diagram of an up-scaling 

process according to the present invention, and Fig. 4 is a schematic diagram of a 
down-scaling process according to the present invention, where SRC is a memory for 
storing a source picture, and DST is a memory for storing a scaled picture. The size of the 
first line-buffer BUFFER 1 is (m* W old ) bytes and the size of the second-buffer BUFFER2 

25 is (n*Wncw) bytes, where m and n are the number of lines for the first and second buffers 

and depend on the length of the filter used. That is, if the length of the interpolation filter 

is L, then m = L and n = (2*L-1). Furthermore, the function of the buffer BUFFER 1 is to 

store data needed for horizontal scaling as well as to store the decimated picture data. The 

BUFFER1 is not necessary if the source picture can be randomly accessed from the 

13 
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memory SRC. On the other hand, the function of the BUFFER2 is to store the 
horizontally scaled results as well as to organize the data for vertical scaling. 

Please refer to Fig. 5. Fig. 5 is a flowchart according to the present invention. In step 
5 2 1 0, the picture to be scaled is inputted and stored in the memory SRC. In step 220, the 
scaling operation for the picture is determined as being up-scaling or down-scaling. If the 
picture is to be up-scaled, during step 22 1 , rows of picture data are transferred to the 
buffer BUFFER! . If the picture is to be down-scaled, during step 222, the data 
decimation operation is determined. If the decimation operation is enabled, rows of the 

1 0 picture data are decimated when necessary and transferred to the buffer BUFFER1 . 

Otherwise, rows of picture data are transferred to the buffer BUFFER1. In step 230, the 
horizontal scaling process is performed and the results are stored in the buffer BUFFER2. 
In step 240, the vertical scaling process is performed and the results are outputted to the 
memory DST. In step 250, if the picture has data remaining to be scaled, repeat step 220 

1 5 to step 240 until all rows of the picture data have been horizontally and vertically scaled. 
Finally, the step 260 is executed to finish the scaling process. 

Please refer to Fig. 6. Fig. 6 is a flowchart of a preferred embodiment according to 
the present invention. For clearly explaining the spirit of this invention, the preferred 
20 embodiment of this invention exploits a 4-tap interpolation filter to perform the scaling 
process. Furthermore, the up-scaling ratio is assumed to be up to twice as large as the 
original picture in each dimension, but the down-scaling ratio is not limited. Therefore, 
the size of the buffer BUFFER 1 is 4* W OJ d bytes and the size of the buffer BUFFER2 is 
7*Wncw = 14*Woi d bytes, as Wncw = 2*W oJd . For conveniently explaining the processing 

m 

25 steps in the flowchart, the whole process can be divided into three groups, namely the 
initialization steps, horizontal scaling steps, and vertical scaling steps. 

In the group of initialization steps, the picture to be scaled is firstly inputted and 
stored in the memory SRC. After that, the scaling factors and the decimation factors are 

14 
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determined. Initially, the horizontal and vertical scaling factors are respectively given by 
Sh = Woia/Wncw and S v = H old /H M w> where W 0 ! d and H 0 w are respectively the width and 
height of the input picture, and and H new are respectively the width and height to be 
scaled to. The initial horizontal scaling factor S h determines whether the picture data will 
5 be decimated in the horizontal direction before being transferred to the buffer BUFFER 1 . 
That is, if the decimation operation is enabled and Sh> 2, then picture data is decimated 
horizontally according to the horizontal decimation factor dh = floor(log2Sh), where the 
floor function truncates the decimal portions; otherwise, no horizontal data decimation 
will be performed and set d h = 0. Similarly, the initial vertical scaling factor Sv 

10 determines whether the picture data will be decimated in the vertical direction before 

being transferred to the buffer BUFFER1. That is, if the decimation is enabled and S v > 2, 
then picture data is decimated vertically according to the vertical decimation factor d v = 
floor(log 2 S v ); otherwise, no vertical data decimation will be performed and set d v = 0. As 
a result, according to the picture decimation process, the data in the memory BUFFER1 

15 and the memory SRC can be generally expressed as: 
BUFFER l(x,i) = SRC(2 dh +x,2 dv *(y+i)), 

where x ranges from 0 to (W 0 id-1), y ranges from 0 to (Hoid-1) and i ranges from 0 to 3. 



Since after decimation the width and height of the input picture will become W d - 
20 Wouj/dh and Hd = Hoi d /d v , the actual scaling factors used in the scaling process are referred 

to as Sh = Wd/W ncw and S v = Hd/H^. After determining the decimation and scaling factor, 

the initial horizontal scaling process is preformed. In this process, two lines of SRC data 

are transferred to the buffer BUFFER 1 , such as: 

BUFFERl(x,0) = SRC(2 dh *x,0), 
25 BUFFER 1 (x, 1 ) = SRC(2 dh *x,2 dv ) t 

where x ranges from 0 to (W d -1). 

Based on Eq. 1, the initial horizontal scaling process on these two lines of data becomes: 

2 

BUFFER2(x2 y 5) - J BUFFER\(x\ + /7,0) • w(n) , and 

n — } 

15 

♦ 
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2 

BUFFER2(x2 9 6) - J BUFFERl(x) + * w(«) 

AT—— I 

where x2 ranges from 0 to (W ne ^l) and xl = floor(S h *x2), which neglects fractional 
portions. The result of the initial horizontal scaling from rowO and rowl of the buffer 
BUFFER 1 is respectively stored in row5 and row6 of the buffer BUFFER2. The practical 
5 implementation of the scaling process and the method of obtaining w(n) are clear to those 
skilled in the art and will not be discussed here. However, one thing that should be 
mentioned is that the boundary conditions are treated as a mirror-extension process. 

Please refer to Fig. 7. Fig. 7 is a schematic diagram of mirrored data extension. 
10 There are two options, and each option uses one row of the buffer BUFFER 1 with the 
width of W as an example. 
In option 1 : 

BUFFERK- 1 ) - BUFFER1 (0), 
BUFFER 1(W) = BUFFER l(W-l), and 
15 BUFFER 1(W+1) = BUFFER l(W-2). 
In option 2: 

BUFFER 1(-1) = BUFFER 1(0), 
BUFFER i(W) = BUFFERl(W-l), and 
BUFFER 1 ( W+ 1 ) = BUFFER 1 ( W-2). 
20 Finally, in the last step of the initialization steps, the vertical index pointers y and y2 are 
both set to 0, where y is the vertical index counter for the memory SRC and y2 is the 
vertical index counter for the memory DST. The functions of pointers y and y2 will be 
clear in the following discussion. 

25 In the group of the horizontal scaling steps, the first step is to re-arrange the buffer 

BUFFER2 as follows: 

BUFFER2(x2,0) = BUFFER2(x2,5) if y = 0, 

16 
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BUFFER2(x2,0) = BUFFER2(x2,4) ify ^ 0, 

BUFFER2(x2,l) = BUFPER2(x2,5), 
BUFFER2(x2,2) = BUFFER2(x2,6), 

BUFFER2(x2,5) = BUFFER2(x2,4) ify ^ (Hd-7), 

5 BUFFER2(x2,6) = BUFFER2(x2,3) ify > (H<p7), 

where x2 ranges from 0 to (W new -1). 

After that, more rows of picture data are transferred from the SRC to the buffer 
BUFFER 1, such as: 

BUFFER l(x,0) = SRC(2 dn *x,2 dv (y+2)) ) 
1 0 BUFFER1 (x, 1 ) = SRC(2 dn *x,2 <JV (y+3)) i 

and ify < (H d -7), then 

BUFFERl(x,2) = SRC(2 dn *x,2 dv (y+4)) ) 

BUFFERl(x,3) = SRC(2 dn *x 3 2 dv (y+5)), 

where x ranges from 0 to ( W d - 1 ). 
1 5 The horizontal scaling process is given as follows: 



BUFFER2(x2,3) - J BUFFER\(xl + w,0) * w(n) , and 



n — 1 



BUFFER2(x2A) - J BUFFER\(x\ + n;\) * w(n) ; 



n — 1 

and if y < (H<r7), then 



2 



BUFFER2(x2,5) - £ BUFFER\(x\ + w,2) * w(rt) , and 



/»— i 



20 BUFFER2(x2,6) - J BUFFER\{xl + *,3) * iv(w) , 

where x2 ranges from 0 to fWWl) and xl = floor(S h *x2). That is, the horizontal scaling 
results from rowO and row I of the buffer BUFFER 1 are saved to row3 and row4 of the 
buffer BUFFER2, respectively. Also, if y is smaller than (H d -7), then the scaling results 
from rbw2 and row3 of the buffer BUFFER 1 are saved to row5 and row6 of the buffer 

17 



PACE 22733 " RCVD AT 5/9/2005 4:24:50 AM [Eastern Daylight Time] * SVR:USPTO-EFXRF-1/0 * DN!8:8729306 * CSID:8064986673 ' DURATION (mm-ss): 13-56 



09-05- '05 16:37 f&- WINSTON HSU, NO. 41526 8064986673 T-599 P23/33 U-605 



AppL No. 10/605,745 

Amdt. dated May 09, 2005 

Reply to Office action ofJanuaTy 10, 2005 

BUFFER2. 

In the group of vertical scaling steps, the first step is to let y I = floor(Sv*y2). After 
that, if yl satisfies y ^ yl< (y-h4) and y2 satisfies y2 < H„ew, then the vertical scaling 
5 process is performed as follows: 

2 

DST(x2, y2) - J BUFFER2(x2, y\ + n) * w(n) , 

»— i 

( where x2 ranges from 0 to (W new -1). After the vertical scaling process, the vertical index 
pointer y2 is incremented by 1, i.e. y2 = y2+l. Finally, the process returns to the first step 

* 

of the vertical scaling steps. On the other hand, if yl does not satisfy y ^ yl < (y+4) or 
10 y2 does not satisfy y2 < H ncw , then the vertical index pointer y is increased by 4, i.e. y = 
y+4. After that, the range of y is verified; if y is smaller than (Ha-3), then the entire 
scaling process begins again from the first step of the horizontal scaling process. 
Otherwise, the scaling procedure has been accomplished and all processes are terminated. 

1 5 In contrast to the prior art, the present invention provides a line-buffer scaling 

method that enables horizontal-scaling and vertical-scaling processes to use the same 
number of filter taps for achieving the best possible quality, while the buffer memory 
requirement is kept to be as small as possible and is proportional to the filter length. 
Therefore, this low-memory scaling method is suitable for the case when system memory 

20 available is constrained. 

Those skilled in the art will readily observe that numerous modifications and 
alterations of the device may be made while retaining the teachings of the invention. 
Accordingly, the above disclosure should be construed as limited only by the metes and 
25 bounds of the appended claims. 
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